The man-machine interface is a primary limitation in integrating machine capabilities for communication, recording, power, and assistance into everyday life. With new and old machine form factors, new user interface systems are made possible. The advent of high definition, small low cost cameras has made it possible to use cameras as part of how a user communicates with a machine. While users are able to learn special behaviors and gestures to communicate to a machine through a camera, the interface feels more natural when a user is able to communicate with the machine as if the other machine is another person.
Many computer input and control systems are being developed which respond to hand motions and gestures. Rather than typing, pressing buttons, or operating a cursor control device, the user makes hand motions in front of a camera. Simpler systems respond only to hand waves and arm motions. For more detailed control, the movements of individual fingers are tracked.
In some systems, a depth-based hand tracking system is used. Different camera systems obtain the depth information in different ways. One such camera system uses two or more cameras physically spaced apart and compares simultaneous images to determine a distance from the cameras to the hand. Other camera systems use a rangefinder or proximity sensor either for particular points in the image or for the whole image such as a time-of-flight camera. A camera system with multiple sensors determines, not only the appearance of the hand, but also the distance to different points on the hand.
One important way in which people communicate visually is through pointing. Hand gestures allow a user to behave in a way that feels natural. However, it is not always easy for the machine to see and observe the hand gestures through a camera. With a head mounted display (HMD), hand gestures are even more useful and are already used in some systems to communicate user intent into the virtual world presented by the HMD. One such hand gesture is pointing. Other gestures include waving, and directing a palm towards or away from the camera.